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DETAILED ACTION 

1 . This office action is responsive to communication filed on December 1 0, 2008. 

Response to Arguments 

2. Applicant's arguments filed December 10, 2008 have been fully considered but 
they are not persuasive. 

3. Applicant argues that there is simply no teaching in Anderson when and how the 
color conversion and white balance integration operations are performed. Other than 
the cited portion which merely describes that the color conversion and white balance 
settings on the raw image data performed, there is simply nothing in Anderson that 
teaches that "the white balance integral processing for the second RAW data by said 
white balance integration device and the color space conversion for first RAW data by 
said image processing device processes are performed in parallel during reading of the 
second RAW data from the image sensing element" as specifically required by claim 16. 

4. The Examiner respectfully disagrees. In response to applicant's arguments 
against the references individually, one cannot show nonobviousness by attacking 
references individually where the rejections are based on combinations of references. 
See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981 ); In re Merck & Co., 800 
F.2d 1 091 , 231 USPQ 375 (Fed. Cir. 1 986). The combination of Nakamura and 
Anderson teaches the white balance integral processing for the second RAW data by 
said white balance integration device and the color space conversion for first RAW data 
by said image processing device processes are performed in parallel during reading of 
the second RAW data from the image sensing element. For instance, Nakamura 
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teaches that the white balance integration is performed during the readout of image 
data from the image sensor into memory (See column 6, lines 21-26), and that the color 
space conversion is subsequently performed on image data read out from memory (See 
column 6, lines 26-36). Nakamura does not explicitly teach first and second areas of 
memory for performing parallel processing. This is taught by Anderson. Anderson 
teaches that second image data is read out from the image sensing element and into 
memory at the same time first image data is read out from the memory for further image 
processing (See column 6, lines 47-56. Anderson states that "while input buffer A is 
filled with image data, the data from input buffer B is processed and transmitted to 
frame buffer B". Anderson teaches that color space conversion is part of the image 
processing that takes place during the transfer from the input buffer to the frame buffer, 
column 6, lines 19-32.). Therefore, as Nakamura teaches that the white balance 
integral processing is performed during the readout from the image sensor and into 
memory, and the color space conversion is performed during the readout of image data 
from the memory, and Anderson teaches that the readout from the image sensor to 
memory and the readout from memory for image processing are performed in parallel, 
the combination of Nakamura and Anderson teaches that the white balance integral 
processing for the second RAW data by said white balance integration device and the 
color space conversion for first RAW data by said image processing device processes 
are performed in parallel during reading of the second RAW data from the image 
sensing element. 
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5. Applicant argues that secondly, the Anderson's so-called ping-pong buffers A, B 
as shown in Fig. 4B are different from the memory of claim 1 6. For example, while the 
memory of claim 16 requires that each of the first and second areas of the memory 
stores the RAW data from the first and second operations alternately, the input buffers 
A and B alternate between an input cycle and processing cycle. See, e.g., col. 5, lines 
52-53. Continuing portion of Anderson (i.e., col. 5, lines 53-57) further describes that 
"[d]uring the input cycle, the input buffers 538 are filled with raw image data from the 
image device 114, and during the processing cycle, CPU 344 processes the raw data 
and transmits the processed data to the frame buffers 536." 

6. The Examiner respectfully disagrees. Although Anderson teaches that during the 
input cycle the input buffers 538 are filled with raw image data from the imaging device 
114, and during the processing cycle, CPU 344 processes the raw data and transmits 
the processed data to the frame buffer 536, Anderson additionally teaches that buffers 
A and B alternate between an input cycle and a processing cycle (column 6, lines 4-14), 
and that while one buffer is in the input cycle the other is in the processing cycle 
(column 6, lines 47-56). 

7. Therefore, the rejection is maintained by the Examiner. 

Claim Rejections - 35 USC § 103 

8. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 
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9. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nakamura et al. (US 6,963,374) in view of Anderson (US 6,847,388) and Taniguchi et 
al. (US 5,619,347). 

10. The Examiner's response to Applicant's arguments, as outlined above, in hereby 
incorporated into the rejection of claim 16 by reference. 

Consider claim 16, Nakamura et al. teaches: 

An image sensing apparatus ("Digital Camera", figures 1-4, column 2, line 56 
through column 4, line 43) comprising: 

an image sensing device ("CCD", 303, figure 4) which outputs image data 
obtained by an image sensing element as RAW data (column 3, lines 50-58); 

a memory ("DRAM", 232, figure 4) which has a first area for temporally storing 
first RAW data obtained in a first image sensing operation of said image sensing device 
and for temporally storing second RAW data obtained in a second image sensing 
operation next to the first image sensing operation of said image sensing device (See 
figure 7, column 7, lines 10-49. First raw image data is written into the DRAM (232) 
over channel 1 . During a subsequent image capture operation, this first raw data is 
read out of DRAM (232) over channel 2 for image processing, as second image data is 
written into the DRAM (232) over channel 1 .); 

a white balance integration device (211a, figure 6A) which integrates at least one 
of the first and second RAW data for a white balance processing (The white balance 
integration device (21 1 a) is part of the image signal processor (21 1 , figure 4). The 
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RAW image data is subjected to white balance processing, and then stored into the 
DRAM (232), column 6, lines 21-26.); 

an image processing device (21 1 , figure 4) which performs image processing of 
the first and second RAW data readout from said memory (The image processing 
device (21 1) performs processing such as color space conversion on the RAW data 
readout from memory (232), column 4, lines 8-10, column 6, lines 26-36, column 7, lines 
24-33.), 

a display device ("EVF", 20, or "LCD", 10, figure 4) which displays an object 
image during imaging on the image sensing element (The display acts as a "live view 
display" (i.e. an object image is displayed during imaging), column 3, lines 16-23.); and 

a control device ("main CPU", 21, figure 4) which controls said memory (232), 
said white balance integration device (21 1a), said image processing device (21 1), and 
said display device (20, 10, column 4, lines 1-27. The main CPU (21) comprises the 
image processing device (21 1 ) which contains the white balance integration device 
(21 1a), a bus controller (218) for controlling the memory (232), and a video encoder 
(213) which supplies analog image signals to the display.), 

wherein, in case said image sensing device outputs third RAW data obtained in a 
third image sensing operation next to the first and second image sensing operations 
(See figure 7. Steps 1, 8 and 10-12 comprise a continuous loop. Because of this, it is 
clear that third RAW data, fourth RAW data, etc. can be output by the image sensing 
device without altering the camera operation.), said control device (21) controls so that, 
said image processing device (21 1 ) processes a color space conversion for the first 
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RAW data readout from said memory (232) in accordance with start of 
exposure/storage of the second RAW data from the image sensing element (303) in the 
second image sensing operation (See figure 8, column 7, lines 24-33. A frame of raw 
data obtained by an immediately preceding image (i.e. first image data) is read out of 
DRAM (232) and subjected to color space conversion while second image data is 
obtained through exposure and storage of the CCD.), and said display device ("LCD", 
10) displays the object image (See "Live View Display" on the right side of figure 8.) 
after the color space conversion processing for the first RAW data (See "Pc", figure 8) 
and the integral processing (See Readout 2, "Pe", figure 8) for the second RAW data 
(See column 7, lines 24-49. The LCD exhibits a "Live View Display" after the color 
space conversion "Pc" and integral processing "Pe".). 

However, Nakamura et al. does not explicitly teach that said memory (232) has a 
second area for storing the second RAW image data, each of the first and second areas 
of the memory storing the RAW data from the first and second operations alternately, or 
that said memory stores the third RAW data in the first area in which the first RAW data 
after processing of the first RAW data by said image processing device is finished. 
Nakamura et al. further does not explicitly teach that the processing device processes a 
color space conversion for the first RAW data readout from said first area in accordance 
with start of reading the second RAW data from the image sensing element. 

Anderson is similar to Nakamura et al. in that Anderson teaches of a camera 
(figures 1-3) with a memory (figure 4a). Anderson also similarly teaches of reading out 
raw image data from an image sensor (1 14, figure 1 , column 5, lines 59-64), storing the 
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data in a memory (530, column 5, line 59 through column 6, line 3), and subsequently 
performing color space conversion on the image data (column 8, line 59 through column 
9, line 7). 

However, in addition to Nakamura et a!., Anderson teaches that said memory 
(figures 4a and 4b) has a second area (Input buffer, 2, B) for storing the second RAW 
image data, each of the first and second areas of memory stores the RAW data from 
the first and second operations alternately, that said memory stores the third RAW data 
in the first area in which the color space conversion of the first RAW data by said 
image processing device is finished, and that the processing device processes a color 
space conversion for the first RAW data readout from said first area in accordance with 
start of reading the second RAW data from the image sensing element (See figure 4a, 
column 4, line 59 through column 6, line 3, column 6, lines 38-56, column 8, line 59 
through column 9, line 8. Anderson teaches, "Referring again to FIG. 4B, the ping- 
pong buffers are utilized during live view mode as follows. While input buffer A is filled 
with image data, the data from input buffer B is processed and transmitted to frame 
buffer B. At the same time, previously processed data in frame buffer A is output to the 
LCD screen 402 for display. While input buffer B is filled with image data, the data 
from input buffer A is processed and transmitted to frame buffer A. At the same time, 
previously processed data in frame buffer B is output to the LCD screen 402 for 
display." As one buffer (i.e. the second area) is filled with raw image data (i.e. the start 
of reading of the second RAW data from the image sensing element), the other buffer 
(i.e. the first area) is emptied and processed (i.e. first RAW data is readout from the 
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first area), which processing involves color space conversion (See 612, figure 7, 
column 8, line 59 through column 9, line 8). Anderson teaches that the input buffers A 
and B alternate between an input cycle and a processing cycle, column 6, lines 8-10. 
Therefore, buffer A is an input buffer during a first phase, an output buffer during a 
second phase when second RAW data is written into buffer B, and again an input 
buffer during a third phase when third RAW data is output from the sensor an into 
memory. Thus, the third image data is stored in the same area (i.e. buffer A) as the 
first image data during the third phase, after the first image data is processed during 
the second phase.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to use ping-pong buffers as taught by Anderson in the 
camera taught by Nakamura et al. to read out raw image data from the image sensor 
concurrent with the processing color space conversion of previous image data, for the 
benefit of improving the display speed of the digital camera and preventing the tearing 
of the image on the display (Anderson, column 5, line 65 through column 6, line 3.). 

Nakamura et al. alone does not explicitly teach that the integral processing 
for the second RAW data by said white balance integration device and the color 
space conversion for first RAW data by said image processing device processes 
are performed in parallel during reading of the second RAW data from the image 
sensing element. However, because the white balance integration taught by 
Nakamura et al. is performed during the raw data writing ("Pe"), and thus in 
parallel with the readout of the second image data (see figure 8, column 7, lines 
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41-49), the combination of Nakamura et al. and Anderson teaches that that 
integral processing of the second image data and the color space conversion of 
the first image data are performed in parallel. This is because Anderson modifies 
Nakamura et al. such that the image processing of the first image data (which 
includes color space conversion) readout from the first area of memory takes 
place concurrently with the readout of second RAW image data from the image 
sensor (which includes integration) and into the second area of memory, as 
discussed above. 

However, the combination of Nakamura et al. and Anderson does not explicitly 
teach a white balance calculation device which calculates a white balance coefficient on 
the basis of the integration result by the white balance calculation, or that the image 
processing is performed in accordance with the white balance coefficient calculated by 
said white balance calculation device. 

Taniguchi et al. is similar to Nakamura et al. in that Taniguchi et al. teaches 
performing white balance (column 1 , lines 9-15) on image data stored in a picture 
memory (1 2, figure 1 , column 9, lines 42-60). Taniguchi et al. also similarly teaches of a 
white balance integration device (15, 16, figure 1, column 9, line 63 through column 10, 
line 8). 

However, in addition to the teachings of Nakamura et al. and Anderson, 
Taniguchi et al. teaches of a white balance calculation device ("white balance coefficient 
calculating unit", 22, figure 1) which calculates a white balance coefficient on the basis 
of the integration result by the white balance calculation (See figure 1, column 10, lines 
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45-60. The white balance calculation device (22) calculates a white balance coefficient 
according to a plurality of factors determined in units 17-21 of figure 1, based upon the 
white balance integration of the white balance integration device (15,16). See also 
column 10, lines 6-44.), and that the image processing is performed in accordance with 
the white balance coefficient calculated by said white balance calculation device (See 
14, figure 1, column 9, lines 50-60, column 10, lines 56-60. Image processing is 
performed by the white balance adjusting unit (14) based upon the white balance 
coefficient calculated by the white balance coefficient calculating unit (22).). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have the image processing device taught by the 
combination of Nakamura et al. and Anderson comprise a white balance calculation 
device and perform white balance processing based upon a calculated white balance 
coefficient as taught by Taniguchi et al. for the benefit of performing a sufficient degree 
of white balance adjustment appropriate to a colored picture without any erroneous 
adjustment or adverse influence due to a high chromaticity region of the colored picture 
(Taniguchi et al., column 2, lines 40-50). 

The combination of Nakamura et al., Anderson, and Taniguchi et al. teaches that 
said white balance calculation device (taught by Taniguchi et al.) calculates the white 
balance coefficient for the second RAW data (see above rationale) while said display 
device displays the object image after the parallel processing of the first and second 
RAW data. Anderson teaches that parallel processing is performed by utilizing two 
buffers, column 6, lines 47-56. Anderson teaches that the image data from one buffer is 
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displayed on the LCD (402) as the image data from the other buffer is processed, 
column 6, lines 47-56. Therefore, an object image is continuously displayed on the 
display, including any time when a white balance coefficient is being calculated. 

1 1 . Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Nakamura et al. (US 6,963,374) in view of Anderson (US 6,847,388) and Taniguchi et 
al. (US 5,619,347) as applied to claim 16 above, and further in view of Sasaki (US 
6,961,085). 

Consider claim 17, and as applied to claim 16 above, the combination of 
Nakamura et al., Anderson, and Taniguchi et al. teaches that said display device 
displays the object image after said white balance calculation device calculates the 
white balance coefficient (See claim 16 rationale. Anderson teaches of a continuous 
display.). However, the combination of Nakamura et al., Anderson, and Taniguchi et al. 
does not explicitly teach a defect correction device. 

Sasaki is similar to Nakamura et al. in that image data is collected from the 
image sensor (12, figure 1), preliminary processing is performed to yield first image data 
(see step 41 , figure 7) which is written into a buffer memory (26a, figure 7). Sasaki also 
similarly teaches that the first image data is read from the buffer memory (step 42, 
figure 7) for additional processing (see column 10, lines 1-57). 

However, in addition to the combined teachings of Nakamura et al., Anderson, 
and Taniguchi et al., Sasaki teaches that the apparatus further comprises a defect 
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correction device which corrects a defective pixel portion of image data when the image 
sensing element has a defective pixel (See column 10, lines 21-57. Sasaki teaches that 
the locations of defective pixels are stored in memory, and when reading the data 
output from the buffer (26a), the defective pixels are corrected.), and that a control 
device controls said defect correction device in such a way that said defect correction 
device corrects a defective pixel portion of the image data (See column 10, lines 29-31, 
lines 37-41.). 

Therefore, it would have been obvious to a person having ordinary skill in the art 
at the time of the invention to have a defect correction device for correcting defective 
pixels as taught by Sasaki correct defective pixels in the image processing device 
during the display of the object image as taught by the combination of Nakamura et al., 
Anderson and Taniguchi et al. for the benefit of keeping the influence of a defective 
pixel to a minimum and preserving a high-definition image (Sasaki, column 3, lines 1-5, 
lines 12-16). 

Conclusion 

12. Any objections previously made by the Examiner to the claims are hereby 
removed in view of Applicant's response. 

1 3. Any rejections of the claims under 35 U.S.C. 1 1 2 previously made by the 
Examiner are hereby removed in view of Applicant's response. 

14. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ALBERT H. CUTLER whose telephone number is 
(571)270-1460. The examiner can normally be reached on Mon-Thu (9:00-5:00). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Sinh Tran can be reached on (571) 272-7564. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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/Sinh N Tran/ 

Supervisory Patent Examiner, Art Unit 2622 



